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(57) Abstract 

Embedding a waiermark in an image by changing seieeied DO coefficients (301) in the blocks and macro blocks of coefficients 
which represent the image. The changes in the blocks that comprise each macro block are done in a coordinated manner so that the phase 
of the watermark signal is preserved across the block boundaries. I*hc bit rate of the image signal is preserved by maintaining a count that 
represents the amount that the bit rate has been increased by cti.in^es in coefficients less the amount that the bit rate has been decreased by 
changes in the coefficients. 
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METHOD AND APPARATUS FOR WATERMARKING VIDEO IMAGES 
Field of the Invention: 

This invention relates to stenography, to copy protection and to applying and 
detecting digital watermarks in video images. 

Background of the invention: 

The advent of technology for storing images on digital media has increased the 
need for a method to protec* against piracy, images stored on prior forms of media 
(e.g. VHS, Beta, audiotapes, etc.) are inherently degraded when copied. Images 
stored on digitally encoded media can be copied with no degradation: therefore, 
perfect copies of copies of copies, etc. can be made. 

The introduction of Digital Versatile Discs (DVD) containing movies has created 
increased incentives for both casual and professional unauthorized copying. At the 
movie industry's urging, technology has been put in place to protect against simpie 
duplication of DVD disks using equipment available to unsophisticated consumers. 
This is similar to the protection that exists which prevents one from duplicating a 
VCR tape by connecting together two commercially available VCRs. 

While such protection mechanisms protect against some types of copying, a 
personal computer connected to a DVD device present a much more complicated 
problem. Open architecture devices such as personal computers reproduce the 
signals in the "clear 5 and such devices have many entry points, which can be used 
to duplicate materia" once it is in the "clear". The present invention uses digital 
watermarks to address the aoove described problem. The present invention also 
has other applications 

1 
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I 

2 It is known that to facilitate tne detection of digital watermarks one can insert a 

3 watermark signal that forms a grid. The grid can be used to determine orientation 

4 and scale. With the present invention the data signal and the grid signal are 

5 integrated into a single watermark signal in such a manner that the visual artifacts 

6 introduced by the watermark are minimized. 

8 In applications such as DVD. an important factor thai needs be considered is the bit 

v rate of the bit stream. There are disadvantages if introduction of a watermark into a 

K> bit stream changes the bit rate. For example if images are going to be recorded on 

1 1 a medium such as a DVD disc, increasing the number of bits in the bit stream will 

12 decrease the number of images that can be recorded on a single disk, it is known 

13 that, in general, adding a watermark to a stream of images will increase the number 
l-i of bits in the bit stream. The present invention provides a method and apparatus. 
1> which preserves the bit rate even though watermarks are introduced into the 

h> images. 

is Summary of the invention: 

ly The well-known JPEG and MPEG data compression techniques transform images 

2u utilizing a discrete cosine transform (DCT) which produces a matrix of DCT 

1\ coefficients. These coefficients are arranged into blocks (e.g. into 8 by 8 blocks of 

22 coefficients). The blocks of DCT coefficients are in turn arranged into macro blocks 

* (e.g. into 16 by 16 arrays containing four 8 by 8 blocks). With the present invention 

?.; selected DCT coefficients in each block are slightly increased or slightly decreased 

25 in response to a watermark signal. The changes in the blocks that comprise each 

:v, macro block are done in a coordinated manner so that the phase of the watermark 

:.7 signal is preserved across the block boundaries. By preserving the phase across 
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1 block boundaries, a detectable grid is formed which can be used as an orientation 

2 and scaling grid. 

a The present invention also maintains the bit rate of the image signal. The bit rate of 
the signal is preserved by maintaining a count (referred to as the cumulative change 

6 count) that represents the amount that the bit rate has been increased by changes 

: in coefficients less the amount that the bit rate has been decreased by changes in 

s the coefficients. If at any time the cumulative change count exceeds a pre- 

( ) established limit, coefficient changes that decrease the cumulative change count 

in continue; however coefficient changes that increase the cumulative chance count 

« ! are suspended. The suspension of coefficient changes that increase the 

\'i cumulative change count continues until the cumulative change count falls below 

the pre-established limit The above described process can be described as 

• i selectively changing the intensity of a watermark signal in a bit stream so as to 

15 prevent the entropy of the combined signal from exceeding a pre-established limit, 

if, Brief Description of the Figures: 

i? Figure 1 is a diagram illustrating how the pixels in an image are arranged into biocks 

18 and how the resulting DCT coefficients are numbered. 

19 

?o Figure 2 is a diagram, which shows how the blocks of DCT coefficients are arranged 

2 1 into macro blocks. 

22 

.■3 Figure 3 is a program flow diagram showing how the coefficients in each block of a 
24 macro block are treated so as to preserve the phase of watermark signal in each 
;*5 macro block. 
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1 Figure 4 is a program flow diagram showing how the bit rate in the data stream is 

2 maintained constant. 

4 Description of preferred embodiments: 

5 The well know MPEG (Motion Picture Expert Group) and JPEG (Joint Photographic 

6 Expert Group) image compression techniques use a DCT (Discrete Cosine 

7 Transform) to generate a matrix of coefficients. The preferred embodiment of the 
s invention shown herer: slightly modifies the DCT coefficients (either slightly 

9 increases or slightly decreases the value of the coefficients) so as to embed a 

it? digital watermark in the image Such a digital watermark can later be detected by 

: 1 conventional cross correlation techniques. 
12 

• 3 As illustrated in Figure 1 : the MPEG and JPEG techniques divide an image into 8 by 
(4 8 blocks of pixels. Each block of pixels is then used to generate an eight by eight 

15 block of DCT coefficients. The 8 by 8 blocks of coefficients are divided into "macro 

• 6 blocks' , each of which consist of four of the original blocks. This is illustrated in 

Figure 2. The rows and columns of DCT coefficients in each block are numbered 

ix from top to bottom and left to right as illustrated in Figure 1. The first row and the 

•*) first column are designated as the "0 ,: row and "0" column. 
20 

:i Certain of the DCT coefficients in each block are selected as the coefficients that 

22 will carry a selected bit of the: digital watermark signal. In the preferred embodiment 

23 the three coefficient:: circled in Figure 1 are used to carry the first or "0" bit of the 

24 watermark data signal There three coefficients are modified, that is, either slightly 

25 increased or slightly decreased depending upon the value of the "0" bit of the 

26 watermark data. In a similar manner other coefficients are slightly changed in order 
21 to carry the other bits of the watermark signal. 
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One aspect of the present invention is directed to insuring that the sinusoids 
generated by the changes made to the DCT coefficients are continuous, that is, in- 
phase across the four blocks that constitute each macro block. First, if the 
sinusoids that carry the watermark are continuous across each macro block, there 
will be less edge effects and the watermark will be less visually noticeable. Second, 
the sinusoids which are continuous over the four blocks of each macro block create 
a low level orientation or grid signal. This low level grid signal can be detected to 
determine the orientation and scaie of the watermark. The grid signal can then be 
detecied using the cross coi relation techniques. Cross correlation detection 
techniques are for example shown in copending patent application 08/649.149 filed 
5/16/S6 and in issued pateni patents 5,748,763 and 5,748 ; 783. 

If certain DCT coefficients in adjacent blocks are modified in the same direction, the 
resulting sinusoids will not be continuous across block boundaries. With the 
present invention the changes made to the coefficients of each of the four blocks in 
a macro block are coordinated so that the resulting sinusoids will be continuous 
across block boundaries within each macro block. The changes are coordinated 
using the rules explained below. 

The blocks in each macro block are numbered as shown in Figure 2. Block one is 
considered the base block. The coefficients in this block are changed in a 
conventional way by the associated bits of the watermark signal. Note, the 
following paragraphs relate to how the coefficients which are circled in Figure 1 are 
changed in response to the "0" bit of the watermark. It should be understood that 
other coefficients must be similarly changed to carry the other bits in the watermark 
data. 
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2 In block 1, the coefficients that are circled in Figure 1 are slightly increased or 

~ slightly decreased in response to the "0" bit of the watermark data. In blocks 2, 3 

4 and 4, the circled coefficients shown in Figure 1 are changed in response to the 

5 zero bit of the watermark according to the following rules. 

6 Block 2: invert the direction of the change if the coefficient is in an odd row. 

7 Block 3: invert the direction of the change if the coefficient is in an odd 

8 column 

•) Block 4: invert the direction of the change if the coefficient is in an odd row 

in or if it is in and odd column, but do not invert the direction of the 

I J change if the coefficient is in both and odd row and in an odd column 

! ? 

\? If the above rules are followed the sinusoids generated by the change in the DCT 

u coefficients will be continuous across the boundaries in the four blocks that form. 

I *. each macro block. These sinusoids will be able to be detected using conventional 

io cross correlation techniques and they can be used as a grid to determine the scale 

17 and rotation of the image. The data bits in the watermark will also be able to be 

is detected using conventional watermark detection techniques. Thus, the watermark 

19 data itself is used to form the grid that can be used to determine scale and rotation. 
20 

:: 1 Figure 3 is an overall program flow diagram of the above described aspect of the 

.'.2 preferred embodiment. The system accepts a stream of data that represents 

?.? images. Block 301 is a conventional device or program module that generates DCT 

24 coefficients for the images in the data stream. These coefficients are sent to a 

25 decision unit 302 which sepaiates the data into macro blocks and sends it to units 
7« 311, 312, 313 and 314 depending upon whether the data represents a block 1. 2, 3 
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1 or 4 in a macro block. Units 31 1-313 modify the DCT coefficients in order to imbed 

2 a watermark signal according to the following rules. 

3 Unit 311: modify the coefficients in a conventional manner to imbed 

4 watermark. 

5 Unit 312: invert the direction of the change if the coefficient is in an odd row. 
(. Unit 313: invert the direction of the change if the coefficient is in an odd 

7 column 

s Unit 314: invert the direction of the change if the coefficient is in an odd row 

9 or if it is in and odd coiumn. but do not invert the direction of the 

: <> change if the coefficient is in both and odd row and in an odd column. 

1 ! The output of units 322 to 3 '4 is combined by unit 320 back intc a single data 

!2 stream. It is noted that each of the units shown in Figure 3 could be separate units : 

\' which are either programmed, or hardwired to perform the specified functions. 

14 Alternatively all the function could be performed in a single programmed computer 

15 on a time-shared basis. The unit which generates DCT coefficients is conventional 
in and such units are know and not part of the present invention. 

! 7 

is The previous discussion describes how a watermark can be introduced in the DCT 

'9 domain. It is noted that the durability of the overall watermarking can be increased 

2o by using two watermarks. One watermark can be added by modification of the 

2 J pixels in the original image in the manner as described in US patents 5 : 748,763 or 
22 5.748,783 and then a second watermark can be added by modification of the 

coefficients in the DCT domain as described herein. 

24 

25 Another problem addressed by the present invention is the need to maintain a 

26 constant bit rate in a stream of bits representing a series of images even though 

2.7 watermarks are added to the images. It is noted that MPEG and JPEG systems use 
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variable length codes to represent data, hence, adding watermarks generally 
increases the bit rate of a data stream. Typical a watermark has no correlation with 
the image into which the watermark is embedded, thus embedding an image in a 
watermark produces an image which has a higher entropy than the original image. 
The bit rate of a data stream transmitting an image correlates directly to the entropy 
of the image. 

Typically the number of codes used to code an image, that is, the number of entries 
in the Huffman table of a coded image, is relatively large (e.g. 500) However, the 
changes that occur when a watermark is introduced into an image can be illustrated 
with following simple exampie. Consider a data stream that has only four symbols, 
s1, s2. s3 and s4 r which are encoded as follows: 



Symbol code 



S1 


0 


i S2 


01 


S3 


110 


ISA 


111 



Then consider a data stream as follows: 

Bit stream: 0011010111010 
Decoded stream 0/0/110/10/1 1 1/0/10 
Decoded message: s1, s2. s3. s2. s4, s1, s2 



When a watermark is added to an image the bits in the image are slightly changed 
In the above simplistic illustrative example, in some situation the symbol s2 would 
be changed to the symbol s3 and hence the number of bits in a bit stream, which 
transmits the image would be increased In fact there are mathematical principles 
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i (not explained herein) which show that when a normally distributed watermark (that 

? is, a watermark with a Gaussian distribution) is added to an image, and the image is 

3 transmitted using variable length Huffman codes, the length of the bit stream will of 

4 necessity be increased. 

5 

6 The present invention provides a technique for insuring that when a watermark is 
added to a data stream, the hit rate will be maintained constant. It is noted that the 

x present invention doe.f not violate the above-described mathematical principle. 

l > because with the present invention, some of the redundancy normally used io 

h? watermark images is in certain circumstances decreased. That is in certain 

i i circumstances the intensity of the watermark is decreased. 

12 

!3 With the present invention, the watermark is modified in response to characteristics 

)4 of the image. Thus, to some extent the watermark is correlated to the image into 

15 which the watermark in embedded. In this way a watermark can be embedded into 

lfi an image and the entropy of the combined image and watermark will be 
substantially equal to the entropy of the watermark alone. 

is 

!*> With the present invention, the system maintains a cumulative count of the amount 

20 that the coefficients have been changed to any point in time. That is, the amount of 

21 positive changes less the amount of negative changes made since the beginning of 

22 the bit stream is tracked. This amount is herein referred to as the cumulative 

23 change count. If at any time, the cumulative change count exceeds a pre- 

24 established positive limit, no further positive changes are made. 
25 

_'-> Normally it is only necessary to insure that changes do not increase the bit rate 

27 unduly; however, in some instances it n;ay also be desirable to insure that changes 
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do not unduly decrease the on rate. If this is desired, the same technique as 
described above can be used to insure that the cumulative change amount does not 
exceed a pre established negative limit. That is, if the cumulative change amount 
exceeds a pre-established negative value, positive changes continue in a normal 
manner, but no further negative changes are made. 

The magnitude of the pre-established maximum (and in both a positive and negative 
direction) are established at the values which constitutes the change in bit rate 
which can be tolerated in a particular system. 

Figure 4 is a program flow diagram showing how the data rate is maintained 
constant not withstanding the fact that watermarks are added to the images in the 
data stream. Block 4^3A shows that a limit on the amount of positive changes that 
can be made to DCT coefficients is established and stored. Blocks 403C shows 
that the cumulative change amount is stored. The cumulative change amount is the 
amount of positive changes less the amount of negative changes that have been 
made to coefficients since the start of the data stream. 

The DCT coefficients are calculated in the normal manner as indicated by block 
401 . Likewise the change in aach the DCT coefficients needed to embed the 
watermark is also calculated in the normal manner as shown by block 405. Block 
405 shows that a check is made to determine if the needed change in a particular 
DCT coefficient is positive cr negative. Block 409 indicates that if the change is 
positive a check is made to determine if the maximum allowable cumulative change 
amount stored in block 403A will be exceeded if the change is made. 
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; Blocks 415. and 417 indicate that the coefficients will only be changed, if the 

2 change does not cause the cumulative change amount to exceed the limit in 403A. 

3 Finally as indicated by blocks 425 and 427 : the cumulative change amount in 

4 register 403C is incremented or decremented if a change to the coefficients is in 

5 fact made Block 431 indicates that the coefficients are sent to the output of this 

6 process and they are then transmitted and processed in a normal manner. 

7 

x It is noted that the present invention relates to embedding a watermark in an image. 

') Various know techniques can be used to detect watermarks embedded in images 

in utilizing the present invention. For example techniques such as those described in 

i! US patents 5,748.763, and 5 : 74S.783 or in the "Communications of the ACM :; Juiy 

\2 1998/vol. 41 . No. 7. or in pending US applications serial number 08/746.613 filed 

I 3 11/12/96 and serial number 08/649.419 which was filed 5/16/96 (all of which are 

:4 hereby incorporated herein by reference) could be used. 
15 

16 While the process has- been described above as one where a change is either made 

n or not made, it should be understood that alternatively, the amount of the change 

in could be decreased if the limii in the cumulative change value is being approached. 

19 It is also noted that the system shown in Figure 4 prevents the cumulative change 

2'j value from exceeding a pre established positive limit. Since adding a watermark to 

2) an image generally increases the entropy of the image and since Huffman code 

22 tables are normally constructed such that an increase in entropy result in increased 

23 bit rate, the use of on!y a positive limit is normally appropriate. However, in some 

24 situations : it may be appropriate to tract if the cumulative change amount exceeds a 

25 limit in both the positive and negative directions. Such a check could be added to 
l<i Figure 4 prior to block 427. 
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I 

2 It is recognized that by implementing the present invention, the strength of the 
watermark is in some cases reduced. However, the reduction is not sufficient to 

4 prevent detection of the watermark. The changes made with the above invention 

5 merely lower the intensity of the watermark in a selective manner, thus in some 
<\ instances more processing may be required to detect the watermark. 

s In many systems, each Huffman code covers several symbols. In such systems the 

0 calculation indicated by block 405 is not the change in a single symbol that results 
io from adding a watermark to the image. In such systems the calculation indicated by 

1 i block 405 is a calculation of 'he change that results in the bit string of whatever 

12 combination of symbols used in the Huffman code to represent a symbol, in some 

l." cases the calculation might have to be done over several combinations of symbols 

14 

i? It is also noted that various aspects of the present invention are shown herein in a 

16 single preferred embodiment Other alternative embodiments could use one but not 

all aspects of the present invention. For example the part of the present invention 

is that relates to maintaining bit rate could be used in embodiments which do not use 

19 macro blocks to establish an orientation grid. Likewise the aspect of the present 

20 invention which relates to the use of macro blocks could be used without the part of 

21 the invention that relates to maintaining a constant bit rate. Finally, while the 

22 invention has been shown in an embodiment that inserts a watermark in the DCT 
">." domain, the invention Lould be used in applications where watermarks are inserted 
24 in other domains. 

25 

>6 While the invention has been shown and described with respect to preferred 

27 embodiments of the invention, various changes in form and detail could be made 
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I I claim: 
■*> 

■L. 

1 . A method for adding a multibit watermark to an image comprising, 

4 generating DCT (Discrete Cosine Transform) coefficients representing said image, 

5 said coefficients being arranged in blocks, said blocks being arranged into 

6 macro blocks, 

7 adjusting said coefficients in accordance with the bits of said watermark to embed 
s said watermark in said image., said adjustments being made in a direction 
*) that sinusoids generated by said adjustments are in phase across block 

)«> boundaries in each macro block. 



1 1 

i2 2. The method of claim 1 wherein each of said macro blocks contains four blocks. 



13 

14 3. The method of claim 2 wherein the blocks in each macro block are numbered 

\fi one, two. three and four and wherein the coefficients in each block are arranged in 
rows and columns, alternate of said rows and columns being designated even and 

1? odd rows and columns. 

:s the coefficient in block one being changed directly in response to the bits of said 

!9 watermark, the bits in blocks two : three and four being changed in accordance with 

20 the following rules: 

21 Block 2: invert the direction of the change if the coefficient is in an odd row. 

22 Block 3: invea # he direction of the change if the coefficient is in an odd 
23. column 

::4 Block A: invert ;he direction of the change if the coefficient is in an odd row 

;.s or if it is in and odd columa but do not invert the direction of the 

:6 change if the coefficient is in both and odd row and in an odd column. 
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1 

2 4. A system for adding a muitibit watermark to an image comprising, 

means for generating OCT (Discreet Cosine Transform) coefficients representing 

4 said image, said coefficients being arranged in blocks, said blocks being 

5 arranged into macro blocks, 

;\ means for adjusting said coefficients in accordance with the bits of said watermark 

- io embed said watermark in said image, said adjustments being mace in a 

8 direction that sinusoids generated by said adjustments are in phase across 

9 block boundaries in each macro block, 

1 c j whereby the same changes which represent said watermark form a grid which can 

I i be used to detect the. scale and rotation of said watermark. 

1.2 

!3 5. The system of claim 4 wherein each of said macro blocks contains four blocks. 

14 

15 6. The system of claim 5 wherein the blocks in each macro block are numbered 

16 one, two, three and four and wherein the coefficients in block are in rows and 

\l columns, alternate of said rows and columns being designated even and odd rows 

us and columns, 

19 The coefficient in block one being changed directly in response to the bits of said 

:u watermark, the bits in blocks two, three and four being changed in accordance with 

j. ! the following rules: 

22 Block 2: invert the direction of the change if the coefficient is in an odd row. 
Block 3: invert the direction of the change if the coefficient is in an odd 

24 column 

25 Block 4: invert the direction of the change if the coefficient is in an odd row 

26 or if it is in and odd column, but do not invert the direction of the 

27 change if the coefficient is in both and odd row and in an odd column. 
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! 

2 6. A method of embedding a watermark into an image comprising, 

y. dividing the elements which form said image into blocks and arranging said blocks 

4 into macro blocks, 

5 introducing said watetmark into the blocks in each macro block in an orientation 
r> such that the resulting sinusoids are in phase across block boundaries in each 

■? macro block. 

s whereby said watermark can be used to both carry data and form a grid which can 

) be detected to show orientation of said watermark. 

! o 

; i 7. A method of introducing a watermark into a stream of bits representing variable 
12 length codes without changing the bit rate of said stream of bits comprising. 

!3 maintaining a cumulative change amount which indicates the amount of the positive 

N and negative changes which are made to said codes. 

15 suspending any positive changes if the cumulative change amount exceeds a pre- 

16 established limit, 

i " whereby the bit rate o* said stream of bits is maintained constant. 
IS 

19 8. A system for introducing a watermark into a stream of bits representing variable 
m) length codes without changing the bit rate of said stream of bits comprising, 

21 means for maintaining a count of the amount of positive and negative changes 

'i2 which are made to said codes, 

2} means for suspending any positive changes if a pre-established limit is exceeded : 

24 whereby the bit rate of said stream of bits is maintained constant. 

25 

1:6 

16 
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9. A method for adding a multibit watermark to an image comprising, 
generating DCT (Discreet Cosine Transform) coefficients representing said image, 

said coefficients being arranged in blocks, said blocks being arranged into 
macro blocks, 

adjusting said coefficients in accordance with the bits of said watermark to embed 
said watermark in said image, said adjustments being made in a direction that 
sinusoids generated by said adjustments are in phase across block boundaries in 
each macro block. 

maintaining a cumulative chance count of the positive and negative changes which 
are mace to said codes. 

suspending any positive changes if said cumulative change count exceeds a pre- 
established, 

whereby said watermark forms a grid which can be detected to determine 
orientation and scale of said watermark and the bit rate of said stream of bits is 
maintained constant. 

10. An image processing method that includes steganographically encoding an 
input image to embed a multi-bit code therein, 

generating DCT (Discreet Cosine Transform) coefficients representing said image., 
said coefficients being arranged into primary size blocks, 

arranging said DCT coefficients into macro blocks, each containing four of said 
primary size blocks, 

changing the coefficients in each block to encode therein a watermark in said 
image, 

the direction of change of said coefficients being such that the sinusoids created by 
said changes are continuous across the primary blocks in said macro blocks, 
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whereby said watermark signals can be used to both carry data and to form an 
orientation and calibration grid. 



11. The method recited in claim 10 wherein said primary blocks each contain 64 
DCT coefficients and each macro blocks contains four of said primary blocks. 

12. The method of claim 7 wherein 

negative changes are suspended if the cumulative change amount exceeds a pre- 
established negative amount 

13. The system of claim 8 including 

means for suspending any negative changes if said count exceeds a pre- 
established negative limit. 

14. The method of claim 9 wherein 

negative changes aie suspended if said cumulative change count exceeds a pre- 
established negative limit. 

15. A method of embedding a watermark in a stream of coded symbols that 
represent a series of images by selectively increasing or decreasing the values 
of said symbols, said method comprising, 

establishing a limit on the allowable entropy of said stream of bits, 
suspending said selective increasing if said limit is exceeded. 
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16. A method of embedding a watermark in a series of bit which represent an 
image without increasing the entropy of said image beyond a pre established 
limit, comprising the steps of: 

changing selective bit n of said image to embed said watermark, 
maintaining a cumulative change count of the changes made to embed said 
watermark in said image.. 

suspending changes which cause said cumulative change count to exceed a pre 
established limit. 

17. A method of embedding a watermark in a series of bit which represent an 
image without increasing the entropy of said image beyond a pre established 
limit comprising the steps of: 

maintaining a count which repiesents the change in entropy as said watermark is 
being embedded in said image. 

suspending changes in said image when said count is beyond a specified limit. 
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